Dynamic Path Selection for Source Routing in Time-Varying Lossy Networks

This paper addresses the path selection problem arising in multi-hop sensor networks, e.g., for Smart Grids. In such scenarios, a communication system consisting of multiple multi-hop paths with time-varying hops connects a source and destination. To avoid interference and keep energy consumption low, the source can only send on one path and accrue a reward determined by the state of the traversed hops. We provide the first mathematical formulation to the problem of optimal sequential path selection under partially observable Markov decision processes. We unveil an intriguing behavior of the myopic policy, arguably the most appealing known way to tackle this problem. We specifically prove under positively correlated hops, that this policy can get locked, i.e., permanently ignores potentially good paths. We also generalize an empirically proven good approach for the single hop case, the Whittle index, and show its intractability for the problem at hand. We propose a new metric, Harmonic Discounted Index (HDI), which (i) circumvents the non-optimal myopic locking and (ii) can be computed efficiently. We evaluate the performance of HDI metric within an index policy in a variety of simulation scenarios and show that routing decisions by the proposed HDI metric outperform those based on alternative index policies.